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DETAILED ACTION 



Claim Rejections - 35 USC § 102 



1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



2. Claims 1-21 are rejected under 35 U.S.C. 102(e) as being anticipated over 
Galanes et al. (US PGPub 2003/0200080). 

As to claim 1 , Galanes discloses a method for unifying speech user 
interface and visual prompt on a display (see paragraph [0010]) comprising the 
steps of: receiving grammar specifying a syntax of at least one speech command 
and having semantic information (see paragraph [0044]); and processing the 
grammar to extract the semantic information for use with both a graphical user 
interface and a speech user interface (see paragraphs [0004], [0012], and 
[0044]). 
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As to claim 2 and 12, Galanes discloses a step of generating graphical 
user interface elements corresponding to the semantic information in the 
grammar. McNair accomplishes the forming of the GUI by the use of audio and 
visual markup control (see paragraphs [0012] lines 1-4; [0049] lines 4-7, and 
[0051] lines 5-13). 

As to claim 3 and 13, Galanes discloses the step of generating speech 
grammar (see paragraph [0051] lines 10-12; [0060] lines 1-3) from the grammar 
(see paragraph [0051] lines 10-12; [0060] lines 18-21) and semantic information. 
The semantic information describes the meaning or significance of the 
information being related between user and multi-modal device by way of 
examples, text, prerecorded audio, audible rendering or speech recognition (see 
paragraph [0060] lines 12-19). 

As to claim 4 and 14, Galanes discloses the method which further uses 
the grammar and the semantic information (like "credit card number") to generate 
visual graphical user interface elements, like labels and textboxes, (see 
paragraph [0061] lines 1-8 and 9-12) required to implement a set of commands 
(like process credit card information) contained in the grammar (see paragraph 
[0061] lines 9 and 10). 
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As to claim 5 and 15, Galanes discloses the method wherein the step of 
processing further comprises processing the semantic information to generate 
semantic directives, "meaningful direction", used for parsing the grammar 
between the graphical user interface and the speech user interface. Galanes 
discloses doing this by providing dialog modules (see paragraph [0049] lines 4-6) 
that with the use of markup languages, which link graphic and audible controls 
(see paragraph [0051] lines 1-6), use extensions, tags and other elements that 
serve as semantic directives to unite the GUI and the SUI (see paragraph [0052] 
lines 1-5). 

As to claim 6 and 21 , Galanes discloses a method for unifying speech user 
interface and graphic user interface commands (see paragraph [0010]), 
comprising the steps of: receiving user entered text via a graphical user interface 
(see paragraph [0061] lines 1 1-13); processing the user entered text via the 
graphical user interface (see paragraphs [0057], [0058]); monitoring the user 
entered text and adding input context to the user entered text (see paragraphs 
[0057] and [0062]); and updating a speech recognizer with the user entered text 
and semantic infomriation of which is done by allowing the user to enter text such 
as grammar (see paragraph [0057]) to be used for audible recognition by the 
system (see paragraph [0062] lines 1-5). 
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As to claim 7 and 17, Galanes discloses updating the speech recognizer 
(see paragraph [0045] lines 8-13) by accepting new text infomiation and input 
context (see paragraph [0062] lines 1-5) to augment and update a speech 
grammar and recognition vocabulary of the speech recognizer (see paragraph 
[0045] lines 8-13). 

As to claim 8, Galanes discloses the method wherein the method further 
comprises the step of updating the graphical user interface (see paragraph 
[0058] lines 5 and 6) once the recognizer is updated (see paragraph [0057] lines 
16-20). 

As to claim 9 and 18, Galanes discloses the method wherein the step of 
updating the graphical user interface (see paragraph [0058] lines 5 and 6) 
comprises the steps of updating graphical user interface directives and elements 
(see paragraph [0052] lines 1-5) to maintain the graphical user interface unified 
with the speech grammar (see paragraph [0051] lines 1-6). 

As to claim 10, Galanes discloses the method wherein the method further 
comprises fomning a window or multimodal environment (see paragraph [0048 
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lines 13-15) for displaying a speech interface command as it is being built using 
the graphical user Interface (see paragraph [0043] lines 7-9), in this way the 
graphic controls can be built with the audible control (see paragraph [0060] lines 
1-15). 

As to claim 11, Galanes discloses a system having a unified speech user 
interface and visual prompt on a display (see paragraph [0010]), comprising: a 
display for providing an output for graphic user interface elements (see figures 1 
and 5); and a processor coupled to the display (see paragraph [0026]), the 
processor programmed to: receive grammar specifying a syntax of at least one 
speech command and having semantic infonmation (see paragraph [0044]); and 
process the grammar to extract the semantic information for use with both a 
graphical user interface and a speech user interface (see paragraphs [0004], 
[0012], and [0044]). 

As to claim 16, Galanes discloses a system having a unified speech user 
interface and graphical user interface (see paragraph [0010]), comprising: a 
display for providing an output for graphic user interface elements (see figures 1 
and 5); and a processor coupled to the display (see paragraph [0026]), the 
processor programmed to: receive user entered text via a graphical user 
interface (see paragraph [0061] lines 1 1-13); process the user entered text via 
the graphical user interface (see paragraphs [0057], [0058]);monitorthe user 
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entered text and adding input context to the user entered text (see paragraphs 
[0057] and [0062]); and update a speech recognizer with the user entered text 
and semantic infomiation of which is done by allowing the user to enter text such 
as grammar (see paragraph [0057]) to be used for audible recognition by the 
system (see paragraph [0062] lines 1-5). 

As to claim 19, Galanes discloses the system wherein the processor is 
further programmed to form visual prompt on a display enabaling controls for 
speech interface such as speech controls, audible prompting, and speech 
recognition (paragraph [0010] lines 3-5). 

As to claim 20, Galanes discloses a machine-readable storage, having 
stored thereon a computer program having a plurality of code sections 
executable by a machine (see paragraphs [0024]-[0027] and [0033]) for causing 
the machine to perform the steps of: receiving grammar specifying a syntax of at 
least one speech command and having semantic information (see paragraph 
[0044]); and processing the grammar to extract the semantic information for use 
with both a graphical user interface and a speech user interface (see paragraphs 
[0004], [0012], and [0044]). 
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Any inquiry concerning this communication should be directed to Josiah 
Hernandez whose telephone number is 571-270-1646. The examiner can 
normally be reached from 7:30 pm to 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Xiao Wu can be reached on (571 ) 272-7761 . The fax 
phone number for the organization where this application or proceeding is 
assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from 
the Patent Application Infonmation Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more infonnation about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 
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